Radiology Reimagined: Advancing Clinical Practice Through AI Innovation (2025)-"Deep Dive on Clinical AI Metrics"

"Deep Dive on Clinical AI Metrics" – Dr. Jason Poff

Pdf Summary

This presentation by Dr. Jason A. Poff, Director of Clinical AI at Radiology Partners, explores critical clinical AI metrics and their application in radiology to optimize AI deployment and oversight. The focus is on understanding, calculating, and utilizing key metrics such as Positive Predictive Value (PPV), Enhanced Detection Rate (EDR), Gain to Pain Ratio (GPR), and Number Needed to Benefit (NNB) to measure AI performance in enhancing radiologist detection of pathologies. EDR, for example, quantifies the additional cases AI helps detect beyond radiologist identification, reflecting AI’s clinical opportunity.<br /><br />The presentation emphasizes a 5-step clinical validation process spanning pre-deployment, point-of-care, and post-deployment stages within a hybrid human-AI oversight framework. Continuous monitoring dashboards track AI performance indicators like positive and negative concordance rates compared to reports, highlighting differences from traditional PPV and NPV measures due to real-world human-AI interactions.<br /><br />Additionally, there is an emerging need for novel metrics to govern hybrid human-AI reporting systems, focusing on radiologist edits, AI bias (automation, complacency, distrust), and overall oversight effectiveness to ensure patient safety and diagnostic accuracy. These post-deployment metrics enable balancing AI benefits with potential risks introduced by AI errors or radiologist reliance patterns.<br /><br />Key takeaways include: using clinically relevant AI metrics to inform deployment decisions and radiologist engagement, integrating ongoing automated and human-in-the-loop monitoring to maintain AI quality, and preparing for the evolving AI drafting era that demands new oversight measures.<br /><br />Dr. Poff encourages radiology professionals to deepen their understanding of these metrics and participate in initiatives through RSNA.org to advance safe, effective clinical AI integration.

Keywords

Clinical AI metrics

Radiology AI deployment

Positive Predictive Value (PPV)

Enhanced Detection Rate (EDR)

Gain to Pain Ratio (GPR)

Number Needed to Benefit (NNB)

Hybrid human-AI oversight

AI performance monitoring

Radiologist-AI interaction

Clinical AI validation process